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ABSTRACT 

The purpose of the American Samoa program. Upgrading 
Language Abilities (ULA) funded under Title I Elementary Secondary 
Education Act, is to raise the English language proficiency of 
educationally deprived high school students to the point where they 
can successfully compete with native speakers in an English medium 
educational system unhindered by inadequate abilities in English. 
This report covers 1973-74 operations. All four high schools in 
American Samoa are involved in ULA. The program is administered by 
the Title I staff through the Secondary Division and the Division of 
Instruction. There are about 1700 students in the program, 36 
teachers and 34 classrooms. Each high school has a teacher resource 
center for the ULA program. ULA students receive 90 minutes of 
instruction daily in addition to their other classes. The scope of 
ULA encompasses several broad areas: classroom instruction for second 
language speakers, curriculum development peculiar to Samoan 
students, special instructional materials for Samoa, and teacher 
training. Teachers are involved in continuous inservice training to 
increase their competencies to teach students who speaks English as a 
second language. Considerable progress was made during fiscal years 
1973 and 1974 in designing a system for cjntinuoucly assessing 
characteristics of the ULA program. (Author/JM) 
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PREFACE 



This document is the 1973-7^ evaluation report of the 
ESEA Title I project, Upgrading Language Ability. It has 
been prepared in accordance with federal government require- 
ments and guidelines relative to ESEA Title I. 

The underlying principle guiding the ULA evaluation is 
to provide useful information to individuals requiring it 
for effective decision making. 

An abstract briefly summarizing the major activities 
and findings of the evaluation has been prepared to give the 
reader an overview of the report. 
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ABSTRACT 



The purocse of the American Samoa Title T Frofrrirr , 
Upgradinc Language Abilities, is to raise the English language 
proficiency of educationally deprived high school students 
to the point v/here they can successfully compete with native 
speakers in an English medium educational system unhindered 
by inadequate abilities in English. The ULA goals and objec- 
tives are consistent with the philosophy and goals of the 
American Samoa Department of Education. 

This evaluation report covers the 1973--7^ project oper- 
ations. The report is addressed primarily to American Samoa 
Department of Ed^ucation personnel, the Title I Advisory 
Council, and the funding authority. 

ULA design and program objectives were based upon several 
needs assessment studies conducted in American Samoa by 
educational experts in language learning and development, 
curriculum and instruction, learning theory, instructional 
media, and teacher training. The design of ULA evolved 
through meetings with parents, community representatives, 
local teachers, and administrators. 

All four high schools in American Samoa — Faga'itua, 
Leone, Samoana, and Manuka — are involved in ULA, The pro- 
gram is administered by the Title I staff through the Secon- 
dary Division and the Division of Instruction. There are 
about 1,700 students in the program, 36 teachers, and 3^ 
classrooms. Each high school has a teacher resource center 
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for the Upgrading Language Abilities program. 'JLA students 
receive 90 minutes instruction daily in addition to their 
other classes. 

The scope of ULA encompasses several broad areas: class- 
room instruction for second language speakers, curriculum 
development peculiar to Samoan students, special instructional 
materials for Samoa, and teacher training. 

Except for the physical facilities, some classroom 
supplies, and counseling and administrative services in the 
high schools, the Upgrading Language Abilities program is 
funded entirely out of ESEA Title I funds. 

ULA materials are still in various developme^ntal stages 
and undergoing constant revision based upon formative feed- 
back from teachers and students. Not all teachers use the 
program materials; some report probT.ems in using them; some 
use commercially available materials and produce their own 
supplementary materials. Teachers are involved in continuous 
inservice training to increase their competencies to teach 
students who speak English as a second language. 

Considerable progress was made during FY 73 and FY 7^^ 
in designing a system for continuously assessing character- 
istics of the Upgrading Language Abilities program. The 
Government of American Samoa Computer Center increased its 
capacity and capability to the point that Instruments can be 
machine scored and analyzed, and data storage and retrieval 
is now available. Several computer programs are on hand to 
facilitate data processing, analysis, storage, and retrieval. 



Descriptive data about ULA students and staff have been 
gathered. 

7he product evaluation examined the effects of seven 
predictor variables on one criterion variable, the English 
Structure Test (EST). The predictor variables included in 
the evaluation were: grade level, language level, use of 
project materials, teacher's degree, teacher's sex, teacher's 
level of certification, and teacher's experience as a ULA 
teacher. Data were analyzed using double and triple analysis 
of variance. It was found that mid level language students 
vho used project materials scored significantly higher 
(p. < .01) than students who did not use the materials. 
Further, it v;as found that 10th and 12th grade classes in 
197^ scored significantly higher than the same grade levels 
for 1973, indicating a growth in language proficiency in 
those two classes. This seems to suggest that a positive 
influence of the ULA instructional variable exists. 

Data analysis also suggests that teacher predictor 
variables of educational level, certification, sex, or 
experience as a ULA teacher are not significantly related to 
pupil performance. 
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INTRODUCTION 



A Title I Program was initiated in 1970 to upgrade 
language abilities (ULA) of secondary students in American 
Samoa. This report discusses the program's background, 
contains a description of its many components, summarizes 
activities and accomplishments for FY 7^ , displays feedback 
and evaluation data relative to program impact, and lists 
recommendations for improving the program. 

Purpose 

The American Samoa Title I program. Upgrading Language 
Abilities (ULA), v;as initiated in 1970 to improve the lan- 
guage abilities of educationally deprived high school stu^ 
dents. Specifically, the goal of ULA is to raise the English 
language proficiency of target students to the point where 
they can compete successfully with native speakers in an 
English medium educational system jnhindered by inadequate 
abilities in English. The students should achieve these 
English language skills without diminishing respect for 
Samoan traditions, culture, and language. The ULA goal and 
objectives are consistent with the philosophy and goals of 
the American Samoa Department of Education . 



•^" Philosophy of the Department of Education . Pago Pago, 
Territory of American Samoa. 1973-7^ 
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The sinrle most difficult problem facing: educators in 
American Samoa has been that of language development. Stu- 
dents grow up in a society that employs tv;o languages — 
Samoan and English. Samoan tends to be the predominant 
tongue used in the home while English is the language used 
in the classroom. As in any similar situation, academic 
progress suffers from this dual language system. 

Teacher skills at both the elementary and secondary 
levels have been generally limited. Because the elementary 
staff is almosi: totally indigenous, students learn much of 
their English from non-native speakers. These teachers 
often lack sufficient training in English to recognize their 
own errors. As a consequence, students entering high schools 
in American Samoa often have been confused by conflicting 
language camples. Although there are many stateside teachers 
in the secondary schools, the shortage of proficient, fluent, 
indigenous teachers persists in that setting also. 

Test scores obtained between 196^ and 1970 indicated 
that impressive gains were made by elementary children in 
their abilities to speak and understand English. The re- 
sults of secondary language instruction were less impressive, 
however. The Science Research Association (SRA) tests 
showed ninth grade students iu 1969-70 to be about three 
academic years below the ability of their mainland counter- 
parts. Twelfth grade students in 1969-70 had slipped to 
about six years behind. V/hile scores based on stateside 



norms probably underestimate the true ability of Samoan 
pupils, the data generally indicated that students were 
making little progress in language ability during their four 
years of high school. 

Three task forces of educational experts in language 
learning and de/elopment, curriculum and instruction, in- 
structional media, and teacher training conducted needs 

2 

assessment studies in American Samoa in 1970 and 1971. 
The design and program objectives of ULA are based on these 
studies and evolved through meetings with parents, community 
representatives, local teachers, and administrators. 



Overview of the Project 

The scope of ULA encompasses several broad areas: 
classroom instruction for second language speakers, curricu- 
lum development peculiar to Samoan students, special instruc- 
tional materials for Samoa, and teacher training. 

All four high schools in American Samoa are involved in 
ULA. The program is administered by the Title I staff 
through the Secondary Division and the Division of Instruc- 
tion. There are about 1,700 students in the program, 36 

^ Report of the Educational Television Task Force . 
Depr.^»tment of Education, Territory of American Samoa. June 
1970. 

Report of the English Language Task Force (I) . Depart- 
ment of Education, Territory of American Samoa. July 1970. 

Samoa Supplementary Report . University of Hawaii-Samoa 
Contract. Department of Education, Territory of American 
.'icimoa. 1971. 



teachers, and 3^ classrooms. Each high school has a teacher 
resource center for the ULA program. In addition to their 
other classes, ULA students receive 90 minutes of instruction 
daily in English-as-a-Second-Language (ESL) and Language 
Arts. Teachers are involved in continuous inservice training 
to improve their competencies to teach students who speak 
English as a second language. 

Some instructional materials for the ULA program were 
purchased stateside. Locally develop „ and published 
materials include a complete handbook for Language Arts 
classes, an extensive series of student exercises cross- 
referenced to the handbook, a series of special texts. This 
Way Up , for the ESL classes, and supplementary exercises for 
this series. In 1973 not all ULA teachers used the program 
materials. Some reported problems using them. The materials 
are still in various developmental stages and undergoing 
constant revision based upon formative feedback from teachers 
and students. 

Except for the physical facilities, some classroom 
supplies, and counseling and administrative services in the 
high schools, the ULA program is funded entirely out of ESEA 
Title I funds. 

Development of instructional materials and inservice 
training continued in 1973-7^. During FY 73, the Title I 
staff developed instruments that were used for product 
evaluation. However, the instruments were available during 
FY 73 to administer as single-group post tests only and, 
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therefore, change in student performance coul'i not be 
measured. "onoiderable progress was made during FY 7^ to 
design a system for continuously assessing characteristics 
of the ULA program utilizing the increased capacity and 
capability of the Governm.ent of American Samoa Computer 
Center. This effort was continued through FY 7^. 
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PROJECT DESCRIPTION 

This section presents descriptive information about 
each component of the Title I ULA program. The following 
components are described: rationale, community, student 
population, organizational structure, project goals and 
objectives, instruction, instructional materials, physical 
facilities and equipment, instructional and administrative 
staff, staff development, and project costs. 

Rationale and Theory 

The learning principle that provides the basis for the 
ULA program is that students can become proficient in a 
second language if they are provided interesting materials, 
motivating classroom activities, and opportunities to 
reinforce what they learn through imaginative and creative 
assignments . 

ULA Student Target Group 

The ULA target group is defined as Samoan students in 
grades 9-12 attending high school in American Samoa, v;hose 
education is hindered by a lack of proficiency in English. 
A student is considered to be ^^hindered by a lack of pro- 
ficiency in P'lnglish" if he fails to meet the standards for 
exemption from ESL defined in the English Language Curricu - 
lum Guide (The Secondary En^^lish Program ) Department of 
Education , American Samoa. 



These standards require a score on a standardized test 
of English Language Proficiency comparable to the scores 
required by United States colleges and trade schools for 
exemption from ESL, reading knowledge of 70 percent of the 
5,000 most frequently occurring English words, and oral/ 
aural proficiency approximately equivalent to United States 
Foreign Service Scales 5-3. Table 1 presents information 
about the students. 



TABLE 1 

Estimated Distribution ESL Students 



Characteristic 


School 


Faga' Itua 
N=311 


Leone 
N='4i|l 


Samoana 
N=672 


Total 
N=1424 


f P 


f P 


f P 


f P 


Grade Level 










9th 


85 


119 


198 


4 02 


10th 


62 


136 


119 


317 


11th 


71 


68 


113 


252 


12th 


-.8 


68 


107 


223 


ESL Levpl 










High 


92 


64 


130 


286 


Middle 


132 


229 


293 


654 


Low 




98 


114 


254 


No response or 










ESL exempt 


ii5 


50 


135 


230 



11 



Most students use English only sometimes during out-of- 
class conversations while only 2 percent reported using 
English often. Twenty-three percent of the students reported 
they seldom used English. Most students are assigned to 
middle level ESL classes. 

Organizational Structure 

The American Samoa Department of Education is respon- 
sible for the free public education of all residents from 
early childhood ulirough post-secondary, adult programs. 
This responsibility is not delegated to villages, countii:^s, 
or local government districts. There are no ''school dis- 
tricts" other than the one whole territory. 

The Department is admiristered by the Director of 
Education and the Territorial Board of Regents, an advisory 
board appointed by the Governor of American Samoa. 

Budgeting and accounting for the Department is done 
primarily by division, including Early Childhood Education, 
Elementary Instruction, Secondary Instruction, Instructional 
Television, and the Community College. The various schools 
are not separately budgeted. 

All recruitment and hiring for the Department Is done 
by the central administrative staff, and the divisions 
assign personnel to the various schools on the basis of need. 

Curriculum in the elementary and secondary schools is 
determined by interdivisional curriculum standing committees 
and published in the form of curriculum guides. Principals 



are responsible for ensuring that teachers follow the pre- 
scribed curriculum. Further uniformity of educational 
opportunity is provided for through regularly scheduled 
meetings of all principals within each division. 

The administration of ESEA Title I Grant Awards is 
greatly simplified because of the above "single district" 
structure of the Department. It is further simplified be- 
cause to date only one project has been funded. That one 
project has dealt with the single most critical need as 
determined in 1970 and confirmed by the Parent Advisory 
Council in 1972. The identified need is for students to 
manifest greater English language proficiency at the secon- 
dary level . 

The position of the ESEA Title I staff in the organiza- 
tional structure of the Department of Education has not been 
clearly dellned. The most appropriate and effective position 
is still a matter of some discussion. Table 2 shows an 
approximation of the roles exercised during FY 7^. 

T itle I A dvisory Council . The basis for involving 
parents and other American Samoans in planning, implementing, 
and asses3in;^' the program in 1973-7^ was the Title I Advisory 
Council. In turn, members of this Council have a responsi- 
bility to encourage the involvement of others and to present 

their views and comments. 

The role of the Title I Advisory Council is to provide 
for active and significant community involvement in program 
planning, implementation, and assessment. The Council is 



TABLE 2 

ESEA Title I Program 
Organizational Chart 



Title I 
Advisory 
Council 



Director 
of 

Education 



Board 
of 
Regents 



Deputy 
Director 
Business 



Deputy 
Director 
Instruction 



Coordinator 
Federal 
Programs 



Assistant 

Director 

Divisions 



Compensatory 
Education 
Project 




Schools 





Direct Supervision 

Cooperation, advice, limited supervision 
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scheduled to meet at least four times a year to carry out 
these roler: : 

1. Provide information concerning educational 
needs. This function was carried out informally 
in the original planning of the program. 
Organization of the Advisory Council provides 
the Department of Education with a method to 
monitor this input continuously and systematically. 

2. Participate in proposal development. The Title I 
plan v;as reviewed by the Advisory Council. 

3. Act as a hearing group to receive suggestions. 
The agenda of Council meetings regularly pro- 
vides the opportunity for interested citizens 
to express their views and suggestions. The 
Council chairman is responsible for recording 
these comments and submitting them to the 
program staff. 

Disseminate information on the program to 
others. Members of the Council provide liaison 
between the program staff and rhe community. 
To fulfill this role, they provide others 
with information by making presentations to 
groups and assisting the staff in answering 
questions about the program. 

Assistant D irector , Secondary Division . Administration 
of the Secondary Division is assigned to an Assistant Direc- 
tor of Education. Administration includes supervising all 
administrators and teachers at the secondary level, reviewing 
content and quality of lessons taught in the secondary 
schools, and supervising curriculum planning and inservice 
training. The Assistant Director, Secondary Division: 
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1. Assists in administering the Secondary 
Division budget 

2. Helps develop TV and classroom programs that 
involve students through discussion and inter- 
disciplinary activities 

3. Makes decisions that affect general and 
specific administration policies of the 
Secondary Division 

4. Maintains direct supervision over the 
activities of the secondary television 
instructors and secondary language specialists 

5. Makes certain that secondary principals, 
classroom instructors, supervisors, and all 
other secondary personnel are on the Job 

6. Sees that secondary schools have adequate 
facilities, materials, and classroom teachers 

7. Assists in determining the scope, sequence, 

and relative difficulty of secondary instruction 

State Coordinator of Federal Programs . It is the 
responsibility of the Coordinator to supervise all aspects 
of federal grant programs in the Department of Education. 
This is a 12-month position under the direct supervision of 
the Deputy Director, Business, 

General areas of responsibilii^y include program plan- 
ning and development, operational analysis, budget control, 
long-range planning, and program evaluation. Specific duties 
include : 

1. Establishing procedures and standards for 
implementation of federally funded programs 
and evaluating compliance to the standards 

21 
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2. Reviewing goals, objectives, activities, and 
procf^dures of proposed federally funded programs 
to correlate them with overall budget require- 
ments and the projection of Department of Edu- 
cation activities as required by federal law 

3. Developing guidelines for continuity and 
consistency in the planning, operation, and 
evaluation of federally supported programs and 
providing a systematic plan for modification 
of existing plans when necessary 

4. Serving as liaison officer in coordination of 
all aspects of federal grants with the Office 
of Administrative Services 

5. Working directly with the Director of Education, 
the Deputy Director of Busint^s^and the Deputy 
Director of Instructional Services, as well as 
the heads of all branches of the Department of 
Education, to develop programs that meet the 
needs and objectives of the overall educational 
proF,ram 

6. Processing all correspondence of federal grant 
nature with appropriate federal agencies 

7. Communicating directly with the Governor of 
American Samoa to receive his evaluation and 
approval as required by federal program 
regulations 

8. Disseminating all information pertaining to 
federal grants to the appropriate personnel 
within the Department of Education as well 
as those outside the Department 

9. Overseeing the expenditure of all funds 
allotted to the Department of Education through 
federal grants; acting in this capacity as 
advisor to program administrators, suggesting 

, " ZZ 17 
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and recommending expenditures to be authorized 
and those to be rejected 

10. Representing the Department of Education and 
the Government of American Samoa in conferences 
and conventions related to federal programs 

11. Acting as liaison between the Government of 
American Samoa and the directors and consul- 
tants of federal agencies located in Washington 
D.C. 

ESEA Title Coordinator . Daily management and super- 
vision of the ULA program is provided by the Title I 
Coordinator. This is a 12-month position under direct 
supervision of the Assistant Director, Secondary Division. 

General duties of this position include program plan- 
ning and evaluation, budget preparation, proposal development, 
reviewing proposals for Title I projects, and ensuring that 
the Title I program remains consistent with goals and 
policies of the Department of Education. Specific respon- 
sibilities of the Title I Coordinator include: 

1. Preparing an annual proposal and budget for 
the ESEA Title I Program 

2. Estimating federal grant expenditures for 
secondary English five years in advance 

3. Requesting the purchase of supplies, materials 
and equipment 

4. Requesting the hiring of personnel as budgeted 
and required by the program 

5. Participating in the parent and staff advisory 
councils as required by the proposal 
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6. Planning the participation of private schools 
in the program with their administrators as 
required by ESEA Title I regulations and the 
prop-ram proposal* 

7. Evaluating the program and disseminating 
information as required by the proposal 

8. Reviewing the manpower requirements of the 
program, ensuring that the goals are met 
efficiently and within reasonable time limits 
in order that ESEA Title I financing can be 
phased out and the funds used for other programs 

9. Supervising and assisting the planning of 
effective curriculum, approach, and methodology 
for teaching English-as-a-Second-Language and 
Language Arts 

10. Supervising and assisting in writing ESL 
materials, a language arts handbook, and the 
preparation of diagnostic and achievement test 
instruments 

11. Arranging for publication of validated mater- 
ials through the Government Print Shop or a 
suitable printing and publishing house 

Supervision includes assigning work, establishing per- 
formance standards, evaluating performance, recommending 
employees to fill vacancies, and ensuring that the operations 
of the program are efficient and economical. The Coordinator 
also provides guidance to approximately 3^ teachers and the 
Title I staff by determining the methodology to be used in 
the classrooms and training situations, by ensuring that the 
materials prepared meet the needs of the students and teachers 
in the classroom, and by advising classroom teachers' 
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immediate supervisors on the basis of classroom observations. 

Title I Central Staff , Four language and language 
learning- specialists made up the original FY 7^ Project ULA 
central staff. Two of these specialists have MA's in ESL, 
one has an MA in English and one has an MA in linguistics. 
Three of the specialists have teaching certificates in 
English or ESL. All four of the staff have teaching experi- 
ence ranging from five to eleven years. Three had several 
years teaching experience in ESL before coming to American 
Samoa, and all of their teaching experience has been at the 
secondary level or above or in ESL, 

The responsibilities of the staff are divided. The 
teacher trainer upgrades the language teaching abilities of 
the secondary teachers and prepares them to teach the 
secondary curriculum. The Language Arts Specialist writes, 
publishes, and disperses a Language Arts Handbook that in- 
cludes objectives, exercises and instructional software for 
grades 9-12. The two ESL Specialists wr^-te, revise, publish, 
and disperse the materials for the ESL classes, grades 9-12, 
including .the series of ESL texts. This Way Up . In December 
1973, the Language Arts Specialist left the project. Her 
position was not filled. As a result, some project activi- 
ties were altered, and some were cancelled. 

Project Goals and Objectives 

The goals and objectives of the American Samoa Depart- 
ment of Education provided the philosophic basis for 
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designing the Title I ULA program. The Department is com- 
mitted to helping each person develop as an individual who 
has the skills*, attitudes, and knowledge necessary to be a 
fluent, literate bilingual in Samoan and English, and to 
have respect for Samoan traditions and culture. Specific 
goals for ULA include: 

1. Correct individual areas of deficiency in 
English language competence 

2. Raise the level of proficiency in the English 
language enabling the student to read and 
express more sophisticated thoughts and ideas 
in English and to cope with the English medium 
of Instruction in other courses 

3. Develop language related study skills, such 
as research skills , outlining, note-taking, 
and dictionary use 

^. Develop written and oral self-expression 
techniques 

Nine long-range objectives were adopted to guide ULA 
during the 1973*-7^ school year. Program objectives .•'nclude: 

1 . Reading 

By 197fi, 60 percent of graduating Samoan secondary 
students, after reading selections from stateside 
high school texts, will be able to answer basic 
comprehension questions on the material indicating 
a reading comprehension level of at least adult 
basic literacy. 

2 . Listening 

By 1978, 40 percent of gradur.ting Samoan secondary 
students in the English-as-a-Second-Language (ESL) 
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program, when exposed to a native speaker of English 
lecturing on material from a stateside secondary 
course, and to two or more native speakers discus- 
sing that topic, will be able to answer basic 
comprehens-^ on questions -on the material. 

3. Speaking 

By 1978, 40 percent of graduating Samoan secondary 
students In the ESL program, when tested by two or 
more native speakers of English, will achieve an FS 
level 5-3 score or equivalent on a similar measure- 
ment Instrument. 

i| . Writing 

By I97B, 40 percent of graduating Samoan secondary 
students when gl^en a topic familiar to them, will 
be able to compose a 25-word paragraoh correctly 
punctuated and organized and sufficiently free of 
grammatical errors to be readily understood by a 
native speaker. 

5. Attitude 

Each year, beginning In 1975, tests on student 
attitudes toward Samoan language and culture will 
indicate no decrease in scores from the previous 
year. 

Process Objectives include: 

1 . Test Development 

By June 1974, a battery of Samoan culture-based 
tests will be purchased, written, or adapted to 
measure student achievement in reading comprehen- 
sion, sneaking, and listening . 
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2 . Inservice 

By September 1973, an inservice teacher training; 
program will be instituted to upgrade teachers' 
language teaching methods (ard knowledge of the 
theoretical base of those methods). 

3 . Materials Usage 

A review of lesson plans and classroom observations 
will reveal that at least 75 percent of the project 
teacher?^ have appropriately applied project mater- 
ials and techniques in the classroom. 

ULA product objectives were analyzed in two v;ays: 
(1) their relationship to the program goals, and (2) their 
technical completeness as objectives according to specified 
criteria. Tables 3 and h display analyses of the ULA objec- 
tives . 

Instructional Program 

Instruction refers to the content, methodology, and 
organization of learning activities in the ULA Program. The 
content builds upon the language instruction students 
experienced in elementary schools. Instruction, which is 
in cv;o parts, is designed to assist underachieving students 
attain the English language objectives previously listed in 
this report. 

One part is an English-as-a-Second-Language (ESL) class 
consisting of a ^I5~niinute period of instruction daily and 
concentrating on ^'language competence . Content of ESL 
classes includes that part of language development which is 

^ 28 23 



TABLE 3 



Analysis of Relationships Among 



ULA-DOE Goals and Product Objectives 



ULA-DOE Goals 


Reading 


Listening 


Speaking 


r 

V^riting 


A. Correct individual 

areas of deficiency in 
English language compe- 
tence . 


X 


X 


X 


X 


B. Raise the level of pro- 
ficiency in the English 
language enabling the 
student to read and 
express more sophisti- 
cated thoughts and ideas 
in English and to cope 
with the English medium 
of instruction in other 
courses . 


X 


X 


X 


X 


C. Develop language re- 
laced study skills, 
such as research skills, 
outlining, note-taking, 
and dictionary use. 


X 


X 


X 


X 


D. Develop written and 
oral self-expression 
techniques 






X 


X 


E. Kelp each person develop 
as an individual who has 
skills, altitudes, and 
knowledge necessary to 
be fluent, literate, 
bilingual in Samoan and 
English, and to have 
respect for Samoan 
tradition and culture. 


X 


X 


X 


X 
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TABLE 4 

Technical Analysis of ULA Objectives 
Accordlnp; to Specified Criteria 



^1 — 1--- , 




Criteria | 


Objective 


Performer 
Identified 


Behavior 
Identified 


Condition 
Specified 


Criterion 
Level 
Established 


Measuring 

Method 
Identified* 


1 


X 


X 


X 


X 


X 


2 


X 


X 


X 


X 


X 


3 


X 


X 


X 


X 


X 




X 


X 


X 


X 


X 


5 




X 




X 


X 


6 




X 






X 


7 




X 




X 


X 


8 


X 


X 


X 


X 


X 


9 


X 


X 


X 


X 


X 



^Measuring methods, while not Included In the written 
objectives, have been specified In the evaluation design. 
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not taught to native ^^peakers of English: the phonology, 
morphology, lexicon, semantics, and syntax which a native 
speaker knov;s. 

ESL content can be divided into two subparts. One 
subpart emphasizes listening and reading comprehension: 
comprehension of spoken and written English and recognition 
of correct and appropriate English usage. The second sub- 
part focuses on oral production: the expression of students' 
ideas and feelings in -jorrectly spoken English. 

The content of in.>truction presupposes that students 
have been exposed to considerable English language instruc- 
tion, and that they already can succeed to some extent in 
communicating in English. The problem being dealt with is 
incomplete comprehension and understanding, i.e., failure to 
detect distinctions Intended by the cpeaker or writer, or 
failure to comprehend more than is conveyed by the words 
alone . 

Students who master the ESL portion of the program, 
or who demonstrate the required proficiency by examination, 
do not participate in this phase of Instruction. 

The second part of Instruction consists of the Language 
Arts classes. One 45-rnlnute period of Instruction daily is 
concerned with Language Arts content which is normally 
taught to native English speakers. This part of instruction 
deals specifically with secondary English Language Arts 
goals. Content helps students develop reading comprehension, 
language-related study skills, research skills, outlining, 
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note-taking, and dictionary use. In addition, students are 
provided opportunities to develop written and oral self- 
expression tecnniques . 

Methodology . Instruction also includes the teaching 
activities, the types of interactions, and the learning 
principles underlying the ULA Program. ULA classes are 
predominantly teacher directed. Teachers use a variety of 
spoken and written activities ranging from controlled and 
directed use of language to free communication. These 
activities include teacher presentation of the language 
objective, student oral practice, question/answer sessions, 
listening comprehension exercises, reading, writing, dis- 
cussions, role playing, etc. The focus of the learning 
process is on learning through use of language rather than 
on learning through the application of language rules. A 
common learning cycle is for a concept to be introduced 
through teacher demonstration followed by an activity that 
allows the student to apply his knowledge. 

TextBooks and workbooks are available for all students, 
and supplementary materials are available in each school. 
Each school has its own teacher resource center. 

Organizatio n. ULA classes are organized by grade level 
with aproximate^y 20-25 students per class. Registration 
is managed by the individual school with assistance from the 
Title I staff. Target students progress through semester 
courses. Provision is made for ESL students to matriculate 
out of the ULA program. 
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Target students attend two ^5-mlnute classes per day of 
special Instruction. One period Is primarily devoted to 
basic English competence In speaking and listening L'^KJili- 
and is referrea to as the English-as-a-Second-Language (ESL) 
class. The second period consists of instruction in language 
application skills, e.g., reading- writing, and is called 
the Language Arts class. 

Instruction Materials 

Instructional materials consist of textbooks, workbooks, 
supplementary books, and other media available for classroom 
use in the ULA Program. Program acquisition, development 
and dissemination of instructional materials fe]'' into three 
general categories during the 1973-74 school year: 

1. Selection and purchase of st^^teside materials. 

2. Preparation and publication of locally 
relevant materials . 

3. Selection and purchase of supportive hardware. 
Stateside purchased materials Included teacher reference 

materials, 3upplt entary readers written in somewhat simpli- 
fied English, educational materials kits, paperback diction- 
aries for students of English, and special materials, e.g., 
word puzzles . 

Locally developed and published materials included a 
complete handbook for Language Arts classes, an extensive 
series of student exercises, cross-referenced to the hand- 
book, continuation of a series of special texts. This Way U£, 

33 

28 



for the ESL classes, and an accompanying handbook for ESL 
methodology including guide to the use of This Way Up and 
supplementary exercises . 

Supportive hardware included continued use of tape 
recorders put into service during 1971-72 . 

Language Arts. The Language Arts Handboo k for teachers 
provides the basis and the main source for the classroom 
teacher in the language arts program. Coordinated with the 
Secondary Eng lish Language Currlcu? um Guide , this handbook 
contains the objectives of the program, as well as suggested 
teaching approaches for the objectives. The revised hand- 
book, consisting of 12 units, was distributed to teachers in 
January, 197^. It contains, in addition to the objectives, 
copies of exercises, quizzes, and visual aids for the teacher 
to use in class sets, and references to a library of approxi- 
mately 45 teacher resource books. 

Unit I of the handbook contains appendices listing the 
additional materials available for a teacher's use in class 
sets. These materials Include the following: 

1 . Anthologies 

On several levels, these are readers for 
student use. Some contain study skills 
questions and/or comprehension questions. 
All have teacher ^s guides and answer keys. 

2. Visual materials for overhead transparency 
machines 

These cover reading comprehension and 
study skills 
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3. Skills books 

A variety of skills are included, such as 
study skills, job skills, and word skills. 

Includes SRA kits, remedial reading kits, 
and kits for both reading and writing skills. 
Approximately 15-20 different kits in all. 

5. Class sets of books for teaching reading compre- 
hension and for Improving student interest in 
reading. 

Many titles and levels of reading are 
represented, as well as a wide variety of 
interests. There are both structured and 
unstructured readers. Included are books 
from several sources recommending books for 
second-language learners. There are about 
135 different titles. 

6. Dictionaries 

English - as -a- Second - Language . Beginning in July of 
1972, the development of book three of This Way Up intro- 
duced a shift in emphasis in the instructional methods and 
materials in the ESL classroom. Essentially, the shift was 
away from rule-cen'-ered materials end ac tivities to language 
use materials and activities which emphasize student-to- 
student communication and student-to-teacher communication. 

While usage rules remain a small part of the instruc- 
tional materials, thr- rules function only as a rough intro- 
ductory guide to the correct use of language. Students no 
longer are expected to cite rules to Justify the use of a 
particular structure . 
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The new materials stress the presentation of language 
in discourse form rather than in the form of isolated sen- 
tence structures. Exercises consist of short plays, stories 
and conversations. The exercises are taped to give the 
students access to accurate native-speaker language models. 
Therefore, language learning comes from the observation of 
the language model, follov/ed by language use in actual com- 
munication that is ultimately reinforced through written 
exercises . 

In printed form, the stories, plays, and conversations 
allow the student to make appropriate selections from con- 
trasted grammatical structures. 

The materials focus primarily on the complex verb 
structure of English, introducing the structures in a 
sequenced manner. The sequence is determined by considera- 
tions such as frequency of use by speakers of English (and 
hence immediate need on the part of the students for those 
structures) plus considerations of ease of learning. The 
more easily learned structures are generally taught before 
the more difficult structures. 

In September 1973, Units 1, 2, 5, and 6 were available 
for classroom use. In January 197^ portions of the revised 
Book 3 were made available, in September an ESL handbook 
consisting of methodology, a This Way Up teaching guide, and 
supplementary exercises were distributed. 

Aside from locally developed materials tailored to the 
needs and interests of Samoan students, certain commercially 



available materials are used. These included the Institute 
of Modern Lanp;uage materials, the Educational Development 
Laboratory's vocabulary materials, and a number of short 
plays . 

Physical Facilities 

Project ULA classes are held in the four high schools 
in American Samoa, three high schools on Tutuila and one on 
Ta'u in Manu'a District, during the 1973-74 school year. At 
Samoana Hipch Fchool there are l4 English classrooms; at 
Faga'itua 6; at Leone 12; and at Manu'a 2. 

Each high school has a resource center in the English 
teachers' workroom. The resource centers contain reference 
books on language arts and ESL techniques and methodology, 
sample texts and supplementary teaching materials • Each 
resource center contains over 100 titles. An ESL file of 
cassette tapes of This Way Up exercises is maintained as 
well as a Language Arts file containing class sets of exer- 
cises cross-referenced to the Language Arts Handbook. Each 
resource center also has a ditto machine. 

ULA Instructional Staff 

ULA instructional staff consists of the high school 
English teachers. Information about their educational and 
experience backgrounds was collected from the teachers 
through the form shown in Appendix A. It is estimated 
that the annual turn-over rate is one-half of the teachers 
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in the "^Itle I Program. Information about the ULA instruc- 
tional staff is displayed in Table 5. 

Some conclusions about the ULA instructional staff can 
be drav/n from Table 5. There tend to be slight^ ^ more 
female than male teachers. Of the total group, at least 
eighty-one percent hold either BA or MA degrees compared to 
seventy-two percent in 1972-73. Forty-five percent of the 
instructional staff have degrees with field of study con- 
sidered relevant to ESL teaching while fifty-one percent 
are fully certified to teach. All faculty are under fifty 
years old with sixty-seven percent in their twenties or early 
thirties . 

Ninety percent of the Title I teachers are rated highly 
proficient in English while only seventeen percent are rated 
highly proficient in Samoan. Seventy percent of the teachers 
have taught three years or less in Samoa. Forty-four per- 
cent have four or more years of previous ESL teaching 
experience . 



Staff Development 

Staff development activities consisted of planned 
interactions between the Title T Teacher Trainer and the ULA 
instructional staff. Activities included inservlce work- 
shops, staff meetings, and individual conferences with ULA 
teachers in their schools. The focus of staff development 
activities was to increase ULA teachers* ability to plan 
and conduct instruction effectively and to use new 



TABLE 5 



Title I Teachers 



— . 

Characteristic 


School 


Total 
N % 


Faga'itua 


Leone 


Manu'a 


Samoana 


N 


% 


N 


% 


N 


of 

10 


N 


% 


Age 






















48+ 




































1 


C or 
D.25 


1 


























311-38 


1 


11.11 


2 


16.67 






1 


6.25 


4 


10.26 


29-33 


5 


55.56 


3 


25.00 


1 


50.00 


7 


43.75 


lb 


41.03 


28-24 


1 


11.11 


7 


50.33 






0 


r-f r— r\ 

j7.50 


-1 h 

14 


3::>.90 


below 24 


1 


11.11 














1 


2.5o 


IILLoojLIlg Claua 


1 


11.11 






1 


50.00 


1 


5.25 


3 


7.69 


Sex 






















Male 


6 


66.67 


6 


50.00 


1 


50.00 


4 


25.00 


17 


43.58 


Female 


3 


33.33 


6 


50.00 


1 


50.00 


12 


75.00 




56.42 


missing data 






















rlciI±Ud.± oUd.UU.o 






















Married 


Q 
0 


00.09 


3 


25.00 


1 


50.00 


12 


ft 1— 

75.00 


nil 

24 


61.53 


Single 


1 


11.11 


9 


75.00 


1 


50.00 


ii 

H 


24 .00 


15 


Ofi if 7 
30.4 r 


missing data 






















CMldhood Residence 






















Am. SaTOa 


1 


11.11 


1 


0.33 






d. 


1^ .|:)U 


4 


-1 r\ Of" 

10 .25 


West. Samoa 


1 


11.11 


1 


8.33 












5.12 


IT C! A 


c 
0 


DO. 67 


LO 


03.34 


1 


r— r\ r\r\ 

50.00 


10 


0*:: .50 


27 




Other 
















25 .00 


h 
4 


10.20 


missing data 


1 


11.11 






1 


50 .00 








5.12 


Highest Degree 








8.33 












23.08 


MA 


3 


33.34 


1 


1 


50.00 


4 


25.00 


9 


BA 


2 


22 22 


8 


66.67 






Q 




19 


48.74 


AA 






















Feleti 


1 


11.11 


1 


8.33 










2 


5.12 


None 


2 


22.22 


2 


16.67 






3 


18.75 


7 


17.94 


missing data 


1 


11.11 






1 


50.00 






2 


5.12 


Field of Study 






















ESL 


1 


11.11 










1 


6.25 


2 


5.12 


Eng. 


1 


11.11 


8 


66.67 






3 


18.75 


12 


30.80 


Sec Ed. 














1 


6.25 


1 


2.56 


Communication Arts 




1 


8.33 






1 


6.25 


2 


5.12 


Other 


6 


66.67 


J 


25.00 


1 


50.00 


10 


62.50 


20 


51.28 


missing data 


1 


11.11 






1 


50.00 






2 


5.12 

1 
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1 

Characteristic 


School 


Total 

N % 


Faga'itua 


Leone 


Manu'a 


Samoana 


N 


i 


N 


% 


N 


% 


N 


% 


Certified to Teach 






















No 


l\ 






33.33 


1 


50.00 


8 


50.00 


17 


^13. 59 


Ye? 


l\ 




8 


66.67 






8 


50.00 


20 


51.29 


missing data 


1 


11.10 






1 


50.00 






2 


5.12 


Teaching Experience 






















A. Sec. English: 






















A. Samoa 






















1-2 years 


3 


33.33 


10 


83.33 


1 


50.00 


11 


68.75 


25 


61.10 


3-^ years 






2 


16.67 








31.25 


11 


28.20 


missing data 


2 


22.22 






1 


50.00 








7.70 


B. Years Teaching 






















in Samoa 






















1-3 years 


5 


55.56 


9 


75.00 


1 


50.00 


13 


81.25 


28 


71.80 


M-D years 


2 


22.22 




25.00 






-J 


18.75 


8 


20.51 


missing data 


2 


22.22 






1 


50.00 






3 


7.69 


D. Years Teaching 






















Non-Native 






















Eng. Speakers 






















1-3 years 






J 


ill. 67 






11 


81.25 


22 


56.41 


M-D years 






7 


58. 33 


1 


50.00 


2 


12.50 


13 


33.33 


7-9 years 














1 


6.25 


1 


2.57 


missing data 


2 


22.22 






1 


50.00 






J 


7 6Q 


£ . Years Sec . 






















Teaching Exp. 






















1-3 years 




1^.15 

-T -T . -T _^ 




11.67 


1 


50.00 


10 


62.50 


20 


til 28 


M-5 years 


J 


11 11 


c; 


ill. 67 








11 2^ 


11 
-1- J 


11.14 


7-9 years 






2 


16.66 






1 


6.25 


3 


7.69 


missing data 


2 


22.22 






1 


50.00 






3 


7.69 


F. Total Years 






















Teaching 






















1-3 years 


3 


33.3^1 


3 


25.00 






7 


^3.75 


13 


33.33 


^-5 years 


2 


22.22 


6 


50.00 






3 


18.75 


11 


28.21 


5-7 years 


2 


22. ?2 


2 


16.67 


1 


50.00 


ii 


25.00 


9 


23.08 


7-9 years 


1 


11.11 










2 


12.50 


3 


7.69 


10+ missing data 


1 


11.11 


1 


8.33 


1 


50.00 






3 


7.69 



40 



35 





School 




! 


Characteristic 


Faga'itua 


Leone 


Manu'a 


Samoana 


Total 




N 


i 


N 


i 


N 


% 


N 


% 


N 




Ability to use 






















English 






















Near native or 














ID 








native 


r 
0 


CC Cn 
DD.DY 


12 


T r\r\ r\r\ 

100. ou 


i 


pu . UU 


1 nn nn 
iUU .UU 


io 


Sufxlcient for 






















instruction 


2 


22.22 














2 


5.12 


Sufficient for 






















routine ins true. 






















Insufficient for 






















classroom 






















missing data 


1 


11.11 






1 


50.00 






2 


5.12 


Ability to use 






















Samoan 






















Near native or 




















7.69 


native 




11.11 


2 


16.67 










3 


Sufficient for 




















2.56 


instruction 


1 


11.11 














1 


Sufficient for 






















routine instruc. 






3 


25.00 








25.00 


7 


17.97 


Insufficient for 


















26 


66.66 


classroom 


6 


66.67 


7 


58.33 


1 


50.00 


12 


75.00 


missing data 


1 


11.11 






1 


50.00 


i 


L 


2 


5.12 
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instructional materials successfully. Inservice training 
was provided for thirty-eight classroom teachers in Language 
Arts and Fnglish-as-a-Second-Language during 1973-7^. Tfic 
following section represents a summary of those activities. 

Inservice sessions dealing with methods and techniques 
of ESL teaching and specific suggestions for teaching pro- 
gram materials were held in the schools. 

From June l8 to July 13 the University of Hawaii held 
an extension course here, ESL 580, a methodology class in 
teaching English as a second language involving practice 
teaching, demonstration classes, seminar-type discussions 
and lecture. The Title I Teacher Trainer was responsible 
for the secondary section of this course, and nine secondary 
English teachers participating in the 100 hour program. 

A "general" category of inservice assistance included 
the following: (a) collecting teacher-written classroom 
materials and activities, (b) preparing these materials for 
teachers in other schools, (c) being available at each school 
to take part in meetings with teachers with problems or 
questions concerning teaching techniques and materials, 
(d) determining needs for teacher resource centers in each 
school, (e) developing and maintaining a file on each teacher 
in the project to include background information, schedules, 
materials used, assistance received from the Project ULA 
staff, observation reports, and inservice training received, 
(f) teaching one regular ESL class at Samoana High School 
both semesters to test methods and program materials. A 
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Community College course, ESL Methodology for Secondary 

t achers, wac held at Samoana and Leone High Schools second 

semester. 

Program Costs 

The ULA Program is financed by ESEA Title I funds. 
The Government of American Samoa provides facilities for the 
program. Approximate FY 73 costs as of June 30, 1973 are 
displayed in Table 6. 



TABLE 6 
Project Expenditures* 





'73 
Carryover 


'7^* Grant, 4-3 
of 3/31/7^1 


Total 


Personnel 


$ 137, 'i29 


$ 22,35S 


$159,781 


Travel 




2,482 


7,776 


Contract 


8,222 


15^* 


8,376 


Equipment 


751 




751 


Materials 
8t Supplies 


8,050 


70 


8,120 


TOTAL 


$159,7^16 


$ 25,061 


$184,807 



*This information is based solely on the most 
recent available monthly budget vs. expenditure 
reports prepared by the accounting office. 
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PROJECT EVALUATION 



Evaluation Is used In this report to mean the process 
of systematically collecting, analyzing, and reporting 
3^formatlon that decision makers consider necessary for 
Judging the efficiency and effectiveness of the ULA Program. 

Evaluation Deslp;n 

The evaluation for 1973-7^ specified the collection of 
both process and product Information. 

Process Evaluation . During this phase of the evaluation. 
Information about ULA's management, plans, and Implementation 
procedures was regularly collected and fed back to decision 
makers. The purposes Included Identifying problems In the 
program design or Its Imolementatlon , providing timely 
Information for program managers, and monitoring program 
Imolementatlon. The reason for conducting the process 
evaluation v/as to allow the staff to Improve ULA while the 
program w;^s In progress. 

To facilitate the process evaluation, a Management and 
Evaluation Matrix for Upgrading Language Abilities was de- 
signed that Included process objectives for each of the 
program components, designation of the person responsible 
for each task, completion dates, and space for evaluative 
comments relative to each process objective. 

Process objectives (tasks) are described In Column I. 
The person or group responsible for seeing that the task is 
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completed is specified In Column II. Column III indicates 
the desired date of completion. The evaluation documenta-- 
tion, Column IV, indicates what must be on file as evidence 
(trace) that the task was in fact completed. 

The project director determined the extent to which 
specific project management objectives (tasks) were met at 
each stage of the project's development. With the comple- 
tion of each project task, the director v;as to record the 
disposition of that task in Column V and to maintain a file 
which included necessary documentation specified in Column 
IV and V. Durin:; the final evaluation the evaluator verified 
the completion of each task. Column VI, Projected Modifica- 
tion for Subsequent Planning, includes information to guide 
the revision and refinement of the project. 

The comoleted matrix is included in this report as 
Appendix E, pages 71-98. 

Evaluative Criteria for Product Evaluation . Long-range 
success of the ULA project is based on the increased language 
ability of Samoan secondary students. The e\aluation in 
this context requires linear data and measurement of cumula- 
tive long-rang effects. 

The target population consisted of all 9th, 10th, 11th, 
and 12th grade students in English-as-a-Second-Language (ESL) 
classes attending the four public high schools of American 
Samoa. Descriptive data about the student were previously 
displayed on page 11. 
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Criterion variables . Cognitive achievement was measured 
with the EST developed and refined the previous year. 
Throup:h the evaluation processes during; FY TS? several impor- 
tant developments resulted which enhanced FY 74 's evaluation 
design. For example, the English Structure Test (EST), used 
as a project achievement instrument, was developed, field 
tested, and now reflects acceptable standards of validity 
and reliability; it provides baseline data for the FY 7^ 
evaluation. In addition, a computer-based information sys- 
tem initiated in FY 73 has begun to provide measures of 
cumulative long-range effect of the ULA project. Both of 
these factors added strength to the FY 7^ evaluation. 

FY 7^'s product evaluation examined the effects of 
seven predictor variables on one evaluative criterion vari- 
able, the English Structure Test (EST). The following pre- 
dictor variables were included in the evaluation: grade 
level, ESL level, use of project materials, teacher's decree, 
teacher's sex, teacher's level of certification, and teacher's 
experience as an ULA teacher. 

Data on the grade level and ESL level were collected on 
student Biographical Information Form . Grade level covered 
the four high school grades. The predictor variable 
^'language level*^ consisted of three values: high ESL 
middle ESL, and low ESL. Target students were divided among 
the three values based upon language proficiency grouping in 
each high school. Proficiency grouping decisions were based 
on principal and teacher recommendations and achievement 



test scores. "Language level'* was determined by the ESL 
proficiency group to which each student was assigned for the 
1974-75 school year. High ESL is defined as a student who 
is able to learn from English medium instruction nearly as 
well as a native speaker; middle ESL is defined as a student 
who is able to learn from English medium instruction with 
difficulty and more slowly than a native speaker; low ESL is 
defined as a student who is not able to acquire significant 
learning from English medium instruction. 

Data on the teacher degree, teacher sex, level of 
certification, and experience as ULA teacher come from 
personnel records and through individual interviews conducted 
by the Title I staff. Data concerning teacher usage of 
material with students were derived from each project 
teacher's class plan and from classroom observation by 
project staff. Table 7 reports the data on usage of materials. 



TABLE 7 

Usage of Project Material in QLA Classes 



Usage of Materials 


No. of Classes 


Percent 


High Usage 


20 


33.9 


Medium Usage 


20 


33.9 


Low Usage 


19 


32.2 


Total 


59 


100 .0 



Evaluation Questions . The purpose of developing evalu-- 
ation questions was to avoid the common error made in evalu- 
ating programs by building the evaluation exclusively around 



a set of Drorram objectives. Stenner^^ lists at least two 
deficiencies of this practice. 

1. Analysis techniques must related to evaluation 
questions, not program objectives; and 

2. treating objectives as the foundation for the 
evaluation limits the possibilities for 
providing information and for relating one 
objective to another. 

The project staff and evaluator generated evaluation 
questions by analyzing the classes of information necessary 
to satisfy project information requirements. The task in 
question generation was to analyze all possible combinations 
of three sets of variables (product objective, program 
organization, and target population) necessary to satisfy 
program Information requirements. 

The following were identified as important evaluation 
questions for the Upgrading Language Ability Project: 

1. Are program milestones being attained on 
time? 

2. Do students who use ULA materials gain more 
than those who do not use the materials? 

3. VJhat types of students gain most from the 
propram and what types gain least? 

4. Row is the teacher's educational level 
related to student performance? 

5. How are teacher attitudes related to program 
implementation? 

6. To v;hat degree have teachers appropriately 
applied project materials and techniques in 
classroom instruction? 



A. Jackson Stenner, An Overview of Information Based 
Evaluation : A Design Procedure. Institute for Development 
of Educational Auditing, Arlington, Virginia. 1972. 
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7. What are the costs of the project? 

8. Is there adequate support for ULA from school 
and Department of Education administration? 

9. Is the project staffed with qualified 
personnel? 

10. Is concentrated instruction effective in 
relieving specific deficiencies? 

11. Do teachers find the project-developed and 
project-purchased materials appropriate for 
their teaching skills and their students' 
language proficiency? 

12. Is there a relationship between materials use 
and teacher participation in inservice training? 

13. Is there a relationship between teacher 
participation in scheduled inservice and 
pupil performance? 

14. Do inservice training activities help teachers' 
self-confidence in teaching? 

15. Did the teachers consider the ULA class plan 
helpful? 



Instrumentation Selection 

Table 8 outlines the instrumentation and data collec- 
tion procedures for each evaluation question. Instruments 
are included in the appendix section of this report. 



Data Collection and Analysis of 
Enp;lish Structure Test 

The EST results from May, 1973 provided pre test data 
for all target students grades 10, 11, and 12. An EST pre 
test was administered to all freshmen of the target group 
in October 1973. Title I staff members monitored the test- 
ing procedures in the high schools. Testing was conducted 
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under closely supervised conditions, usinp. the snrr^e i!ir^t!MK*- 
tions and pi^ocedures in each classroom and allowing the same 
amount of testing time. Testing was preceded by inner^vlcc 
training for ESL teachers conducted by the Title I staff. 
Testing proc ^u^'^es and instructions are located in Appendix F 
pages 11^-118. 

Data G^ade Comparison 1973-197^ > The first analysis 
involved a grade ^- p;rade comparison between 1973 and 197^ 
EST results i oro.r to assess possible cognitive growth as 
measured by EST. One-way analysis of variance with repeated 
measures I'/as used. Table 9 displays EST means and sample 
size for all categories :7or each year.* 



TABLE 9 

Evaluation Means for Participant Categories 
Grade by Grade 1973-1974 



Grade Level 



1973 
n m 



1974 
n m 



9th 


368 


25 .20 


512 


28.85 


10th 


^2 


27.i\6 


366 


29.^^5 


11th 


330 


29.61 


313 




12th 


2m 


29.72 


259 


33.95 



Data were gathered both years for the freshmen. How- 
ever, different test forms v;ere used in 1973 and 197^, 
:nerefore, comparisons were not included for freshman groups. 
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Apply Liir the analysis; of variance, a Gipnificant dif^ 
ference (v. < .01) was found to exist between 1973 10th 
r^rades and 197^^ 10th p:rade5 . The 1973 and 197^ 12th cr.rade 
classes were also found to be significantly different (p.c 
•01), Eleventh grades scores for 1973 and 197^ were not 
significantly different. Table 10 reports the analysis of 
variance data for each grade, 

V/hen a significant relationship was found between EST 
evaluation anri one of the categories, the relationship was 
tested further using the Scheffe method which is an overall 
estimation of the strength of a relationship. 



TABLE 10 

Grade by Grade Comparison for 
1973-197^ Students 





Category 


Source 


SS 


df 


MS 




ratio 


10th 


Betwep.n 


71,^^69. 


60 


776 


92. 


10 








Within 


769. 


00 


1 


769. 


00 








Total 


72,238. 


60 


777 






8. 


35** 


11th 


Between 


3'^017. 


07 


6'^1 




27 








Within 


123. 


38 


1 


123. 


38 


1. 


'^7NS 




Total 


il'^,1'^0. 


^5 


6'^2 










12th 


Between 


35,^99. 


27 


'^71 


75. 


37 








Within 


2,096. 


88 


1 


2,096. 


88 


27. 






Total 


37,595. 


09 


^72 











**Significant p. ^ .01 

These findings would seem to suggest a positive influ- 
ence of the instructional variable in grades 10 and 12 
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Evaluat lor. Relative to Several Predictor V ariables . A 
triple classifications analysir of variance was used to test 
for a relationship between acuievenent ac> measured by EST 
ani the follGv;ing predictor var'^ables: gi"ide level, language 
level, teacher^s degree, teacf" s sex, l^^vel of certifica- 
tion, usage of ULA materials with students, and experience 
as an ULA teacher, 

h 

rata were analyzed using Program AV/.R23 . This tech- 
nique permits classification of subjects into levels of two 
or three independent variables simultaneously. Tests of 
significance are computed for the general effects of each of 
the two or three "factors", as well as for their interaction 
effects upon the dependent variable. 

An analysis of variance between achievement as measured 
by EST and predictor variables of materials usage and 
student levels was made. Table 11 displays EST means and 
sample size for each category. 



n 

Fortran Programming for the Behavioral Cc ience , 
Arnold V, Veldman, Holt, Rinehart and Winston, N,Y, 1967, 
P> 257- 
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TABLE 11 

Evaluation Means and Sample Size for Participant Groups 
Lanp:uage Level - Material Usage 



Category 






Learning 


Level (ESL) 








HI 






Mi 


Lddle 


Lov; 


Materials Usage 


n 




m 


n 


m 


n 






m 


High 


52 


36 


.135 


277 


29. 206 


60 




21 


.500 1 


Middle 


96 


37 


.365 


176 


26.375 


37 




22 


.000 1 


Low 




38 


.575 


39 


24.282 


136 




20 


.581 ! 

i 



The analysis revealed an interesting relationship 
(p. < .01) between student achievement as measured by the EST 
and predictor variables of student level and ULA material 
usage. Relationship between student level and EST scores 
was significant. This was to be expected in that the 
student ESL grouping and EST achievement are both related to 
the student language ability. 

In the high level group and the low level group there 
was no significant difference in achievement between high, 
medium, and low users of materials. 

In the middle level group, the students using materials 
scored significantly higher (p. < .01) than students who did 
not use the material. Tab]e 12 reports the triple analysis 
of variance data. 

Figure 1 illustrates the findings in a graphic manner. 
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TABLE 12 

Triple Classification Analysis of Variance 
Sti.dent Level - Material Usage 



Category Source 


SS 


df 


MS 


F ratio 


Between 




8 






A Material Usage 


128.50 


2 


6i< .25 


NS 


B Student Level 


25, 528.i<0 


2 


12,76i< .20 


231.13** 


AB 


910.00 




227.50 


i^.12** 


Within 


i<9,9l8.88 




55.22 




Total 




912 







^^Significant p. < .01 



Analyses were also made of the relationship between EST 
scores, student level (low and middle), and teacher degree 
(MA, BA, non-degree), Tc.ble 13 present s EST evaluation means 
and sample size by category, 

TABLE 13 

Evaluation Means and Sample Size for 
Participant Groups 
Student L-^vel - Teacher Degree 





r 


Feacher 


Degree 






Category 


MA 


BA 


Won 


LEVEL 


n 


m 


n 


m 


n 


m 


Middle 


131 


27.88 


333 


27.71 


125 


27.62 


Low 


57 


21.07 


129 


20. 7^^ 




19.56 
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Table l'^ reports the analysis of variance data. 



TABLE 1^ 

Triple Classification Analysis of Variance 
Student Level - Teacher Degree 



Category Source 


SS 


df 


MS 


Evaluation 


Between 














A Student Level 


7,363 


.79 


1 


7,363 


.19 


118.83** 


B Degree 


76 


.2il 


2 


38 


.12 


0.62 NS 


AB 


iJ3 


.07 


2 


21 


.52 


0.3^ NS 


Within 


7,513 


.95 


817 


61 


.97 




Total 






822 


70 


.69 





Significant p, < .01 



There was no significant relationship found between 
teacher's degree level and achievement by mid and low 
students . 

An analysis of the relationship between ESL achievement, 
teacher certification, and high, mid, and low level of stu- 
dents was made. Table 15 reports EST evaluation of means 
and size of samples by categories. 
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TABLE 15 

Evaluation Means and Sample Size for 
Participant Groups 
Teacher Certification - Student Level 





Category 


Student Level 


High 


Middle 


Low 


Certification 


n 


m 


n 


m 


n 


m 


Certified 


129 


37. 'iO 


3^16 


27.61 


111 11 


20.57 


Non-Certified 


157 


36.il3 


308 


27.^5 


110 


21.21 



There was no significant relationship found between 
teacher certification level (certified/non-certified) and 
student achievement. Table 16 reports the analysis of 
variance • 



TABLE 16 

Triple Classification Analysis of Variance 
Teacher Certification - Student Level 



Category Source 


SS 


df 


MS 


F ratio 


Between 










A Certification 


6.116 


1 


6.116 


NS 


B Student Level 


112,891.011 


2 


21,11113.52 


386.62** 


AB 


107.811 


2 


53.92 


NS 


Within 


65,898.36 


1,188 


55. n 




Total 


108,903.70 


1,193 


91.29 





^"Significant p.< .01 
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There was no significant relationship found between 
student achievement as measured by EST and the predictor 
var*lable3 o!' i3ex (male/female) and teacher experience 
(experience/no experience). The analysis of variance is 
reported in Table 17. 



TABLE 17 

Triple Classification Analysis of Variance 
Teacher Experience - Sex 





Category Source 


SS 


df 


MS 


P ratio 


Between 














A Level 


39,992. 


82 


2 


19,996 


.ill 


364.^3^19** 


B Experience 


22. 




1 


22 


.ill 


0.il082NS 


C Sex 


11. 


35 


1 


11 


.35 


.21 NS 


AB 


19. 


12 


2 


9 


.56 


.17 NS 


AC 


155. 


68 


2 


77 


.81 


1.11 NS 


BC 


2^15. 


01 


1 


2H5 


.01 


1.16 NS 


ABC 


200. 


00 


2 


137 


.89 


2.51 NS 


Within 




90 


l,l8l 


5^ 


.90 




Total 






1,192 


88 


.56 





**Significant p. < .01 
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The relationship betv/een achievement and the predictor 
variables of grade level and student ECL level and school 
was examined at Samoana and Paga^itua. EST evaluation means 
and sample size by category are reported in Table l8, 

Ach"ievenent level at Samoana High School was found to 
be significantly higher (p. < .01) than achievement level at 
Faga^itua High School. However there v;as no significant 
difference between grade level and achievement by low, mid, 
and high students. 



TABLE 18 

Triple Classification Analysis of Variance 
Samoana and Faga^itua - Grade Level 



Category Source 


SS 


df 


MS 


F ratio 


Between 












A 


2056. 


^2 


1 


2056. il2 


39.57** 


B 


^60. 


86 


3 


153.62 


2.96 


C 


2388. 


03 


2 


119^10.17 


229.78** 


AB 


i|87. 


56 


3 


162 .52 


3.12 


AC 


133. 


56 


2 


66.78 


1.29 


BC 


. 


32 


6 


75.72 




ABC 


381. 


06 


6 


63.51 


1.22 


Within 






778 


51.96 




Total 






801 


58.25 





Significant p.< .01 
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CONCLUSIONS AND RECOMMENDATIONS 



The purpose of the FY 7^ evaluation of the ESEA Title I 
project, Upgrading Language Ability (ULA), was twofold: 
one to collect process information about program managemeni: 
and implementation; and two, to assess the extent to which 
the program achieved iT:s goals and objectives, A set of 
evaluation questions i/as systematically developed to provide 
scope and quality to the evaluation design, A discussion of 
each question follows, 

1, VI ere Proj ect Milestones Achieved on Schedule ? The 
evaluator reviewed the Management and Evaluation Matrix with 
the project director and verified the completion of project 
tasks. Data indicate that approximately Q0% of all manage- 
ment tasks were completed as scheduled. Several management 
tasks were not accomplished (e,g,, second semester teacher 
plan utilization and the development of Language Arts hand- 
book unit phase II) because of the departure of the Language 
Arts specialli^t in December, Complete information relative 
to the disposition of intended management activities pre- 
scribed in the evaluation design is detailed in Appendix B, 

Although some management tasks were not achieved, ade- 
quate data were collected to provide conclusions about the 
several components of the Title I program. 
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JO Jtudents Who U3e_ ULA^ Mater ial Gain More Th^.n 
V/?'. \<A, Ufi* M;i^.Lrla!? Mid level students who 



were hl^h iu:orr. of UL/^^ r^r.aterialG achieved significantly more 
than Gtudento not using ULA materials. 

High level and low level students gain least from the 
project. The discovery that low students who use the ULA 
material do net perform better than low students who do not 
use the ULA naterial might be accounted for by the facts 
that 1) the ULA materials presuppose a modicum of facility 
with English and 2) in recognition of this, the project 
provides commercially available materials suited for begin- 
ning learners of English, The discovery that high students 
who use the ULA material do not perform better than high 
students who do not use the ULA material might be accounted 
for by an ability in this proficiency group to learn better 
from heavier ex^oosure to uncontrolled reading materials such 
as are made available for use at the teacher^s discretion. 
It is also net inconceivable to this writer that the language 
proficiency of this group is nearing the level where overt 
language instruction produces greatly diminished returns. 
If this hypothesis we -e to be shown to be correct, a wise 
course of action might be to lower the standards for exemp- 
tion from ESL. 

3. V/hat Type of Student Gains Most From ULA Program, 
and What Ty:;e Gains Least ? As reported in question 2, mid 
level students v/ho were high users of ULA materials achieved 
significantly higher than students not using ULA materials • 

60 
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High level and low level students gain least from the 
projoct. The discovery that low students who use the I'LA 
material do not perform better than low students who do not 
use the UL^ material might be accounted for by the fa. is 
that 1) the ULA materials presuppose a modicum of facility 
with English and 2) in recognition of this, the project 
provides commercially available materials suited for begin- 
ning learners of English, The discovery that high students 
who use the ULA material do not perform better than high 
students who do not use the ULA material might be accounted 
for by an ability in this proficiency group to learn better 
from heavier exposure to uncontrolled reading materials such 
as are made available for use at the teacher's discretion. 
It is also not inconceivable to this writer that the language 
proficiency of this group is nearing the level v;here overt 
language instruction produces greatly diminished returns. 
If this hypothesis were to be shown to be correct, a wise 
course of action might be to lower the standards for exemp- 
tion from ESL, 

H , How Is the Teacher ' s Educational Level Related to 
Pupil P erfornance ? Data analysis suggests that teacher's 
educational level is not signix'^icantly related to pupil 
performance on the EST. Other teacher predictor variables 
including certification (certified/non-certified) and exper- 
ience as an ULA teacher were also found not to be signifi- 
cantly related to student achievement. 
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5- How Are Teacher Attitudes Related to Program 
Implementation ? The project staff reported less utilization 
of project materials by teachers who did not regularly 
attend workshops. This likely affected student achievement 
at least In the mid level classes. 

Teachers did not use complete class plans for second 
semester. Lack of data from this source had a negative 
effect on project monitoring, 

6, To What Degree Have Teachers Appropriately Applied 
Project Materials and Techniques in Classroom Instruction ? 
Data are not available, 

7. What Are the Costs of the Project ? During FY 7^ 
the ULA ESEA Title I project expenditures included $l8^,807 
through March 31, 197^. Budget break outs by cash flow 
objects are reported in Table 6 page 38, 

8, Is There Adequate Support for ULA from School 
and DOE Administration ? School and DOE administration alway 
supported and cooperated with the project in arranging 
meetings, tests, and other requirements involving: personnel. 
One school failed to act on requests by project staff to 
improve security for project equipment. Occasional long 
delays were experienced in shipping supplies to Manuka High 
School but it is not known if these delays were caused by 
negligence on the part of Department of Education warehouse 
personnel or on the part of Government of American Samoa 
Port Facility personnel, 

G7 



9- Is^ the Project Staffed with Qualified Personnel ? 
The project appears to be staffed with personnel who meet 
the established qualifications. 

The Title central staff includes two specialists Wxun 
masters degrees in ESL, one specialist with a Masters in 
English, and one with a Masters in Linguistics, Three of 
the specialists have teachi.ig certificates; all four have 
teaching experience. 

The ULA instructional staff consists of the high school 
English teachers. The Government of American Samoa official 
description of classroom teacher (English) lists the follow- 
ing ninimum qualification, 

0 BA degree 

o Teacher certificate - Master Teacher 
evaluation status 

0 Proficiency in use of English may be 

substituted for the r^egree arid certification 

0 Lxperience: at least one year of successful 
classroom teaching with English as the 
language of instruction 

76% o"^ the project teachers have a BA or above 

55% have certificates 

100% have proficiency in the use of Englisn 

100% have at least one year of experience in English 

10, Is_ Concentrated Instruction Effective in Relieving 
S necj f ic Deficiency ? Process data are incomplete in this 
area. Classroom observations were not completed as scheduled, 
tnerefore, this question cannot be answered, 
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11. ^leachers Find Project Developed Materials and 

Project Purchased Materials Appropriate for Their Teachins 
Styles and Their Students ' Language Prot . clency ? In general, 
yes. The project teachers were asked to respond to various 
questions concerning ULA materials on an interview schedule. 
The Intervlev; form and relevant data are reported In 
Appendix C, Approximately 67% of the project teachers were 
classified as moderate or high users of the material. In 
response to the question "What do you like best about the 
material?", th^re were 66 positive responses by the teachers. 
The follov/lng comments were the ones most often listed. 

o Very interesting (high Interest) 

o Culturally appropriate 

o Exercises good 

o Gramnar is easy to understand 

The most often mentioned comments concerning the ques-- 
tion, "V/hat do you like least about the materials?", were as 
follows : 

c Too l'>ng and difficult (students get bored) 
o :'eed more exercises 
o :io pictures 

There were 36 responses to the "like least" question. 

Data fron another questior. provides ad^^itional inter- 
esting information. Teachers were asked, "If you do rot use 
This V/ay Up in this class, could you please explain why you 
selected other materials?". The most predominate comment 
from teachers in 3ow level groups was that the materials 
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w^r^- ^ilfricuit: for t:.e lov; level / • "edcr.c:'. 

nlu 'i: 'i ^^'•''-^^13 dli r?t: ccnment on the :^est:on. 

12. ^ There a Relationship Between Material s ''300 and 
Teacher Participation in Scheduled Inservice Programs ? The 
oroject Gtarf reported better utilization of project mater- 
ials by teachers who attendee workshops. Unfortunately 
classroom observation data were not systematically collected 
which could be used to corroborate the staff's perceptions. 

13. I_s There a Relationship Between Teacher Partici - 
pation in Scheduled Inservice and Pupil Performance ? The 
question cannot be answered because of a lack of classroom 
observation data. 

14. Do Inservice Training Activities Help Teachers ' 
Self-Conf idence in Teaching ? Data are limited In this area. 
Howe^^er, those teachers who attended workshops were perceived 
by the project staff to use materials more intensively and 
more effectively . 

15 . Did the Teachers Consider the ULA Class Plan 
Helpfu l? The ULA class plans were not used second semester. 
It may be concluded from this that teachers did not consider 
then helpful. 



Recommendations 

The follov;ing recommendations are offered for program 
improvement . 

1. Each of the recommendations stated in the 1972-73 
final evaluation report continues to be valid. Specific 
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attention shoul'I bo r):ilj to those recommendations triat v/ere 
n-' t fr-llo'-V'i c r wh^^r^^ ,T.ln:*r,al orop'^'ess v;as nade. 

2. ProcLiot evaluation reported elsev/here in thi:^ docu- 
ment recor:ne:.ds that '^Specific instructional materials 
should be designed for r?L lov; language proficiency students, 
^his is of particular importance. 

3. Process evaluation of the ULA should be continued, 
utilizing appropriate management tools. FY 75 process 
evaluation cnc uld more effectively collect and feed back 
information on a regular basis, thus providing more timely 
and comtplete data for program monitoring and evaluation. 

^. Mere complete student biographical data should be 
gathered in order to provide information about the extent 
to v;hich ULA project objectives are being achieved v:ith 
dix'ferent groups of students. 

. The difficulties encountered with the 1973-7^ 
planned methods of keeping student records and the resultant 
difficulties in product evaluation on the basis of individual 
pre/post testing m.ake it highly desirable that nev; and 
perhaps innovative student record keeping procedures be 
planned in close cooperation v;lth school and DOE personnel 
and that concerted efforts be made to carry out the plans. 
The possibility that the Title I program could provide sig- 
nificant help and perhaps leadership for the fledgling 
Student Infcrn.aticn Cystemi should be explored. 
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6. In vlev; of product evaluation reported elsewhere in 
this document, consideration should be given to lowerlnc 
the standards for exemption from ESL and concentrating ef-- 
forts on v:hat are currently the mid and low level groups. 

7. The relationship between inservice and impact on 
instruction should be more closely examined, 

8. Systematic classroom observations should be imple- 
mented on a regular basis to assess inservice training needs 
and to provide feedback data to project staff on material 
usage. 

9. The Title I Advisory Council should be more effec- 
tively involved in project planning and assessment, 

10. Objectives for FY 75 dissemination activities should 
be established and an implementation program developed to 
provide information about the Title I program to the Depart- 
ment of Education and the community, 

11. The DOE Title I director and director of the testing 
department are able to utilize the computer in data analysis, 
storage,- and retrieval. As a result of the past two years 
operation considerable descriptive data about ULA students 
and staff are now available. For example frequency distri- 
bution of 1973 and 197^ EST scores converted to percentile 
ranks and T-scores are available to facilitate project plan- 
ning. These data are reported in Appendix G. It is recom- 
mended that steps be taken to assure continued support and 
development in this vital area. 
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APPENDIX A 



ESEA Title I Project ULA 
Teacher Background Information Form 
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ESEA TITLE I PROJECT ULA 
TEACHER BACKGROUND INFORMATION 

Name : 



Today's date: 



Social Security No. 



I I. 



Year of birth: 



Sex: I I 
M F 



to 1520 '2l-'30 '31-'3^ '36-''40 '41-M5 »46-'50 
Marital status: 



Year of birth of 
eldest child: 



Residence during childhood: . . 

Am, Samoa W, Samoa U. S. A. Other 
********************************************************************** 

Highest Academic 
Degree received: 



r 



r 



1 — r 



Ph,D, Master's Bachelor's Associate Heleti None 



Institution: 



Field of Study: 



Academic credit that you have received but for which you have not 
received a degree: 

Institution Field of Study Credit 



Graduate level: 
Undergraduate : 
Non-degree: 
None : 



********************************************************************** 

Are you certified or licensed to teach? 



No 
Yes 



Place certified (North Dakota, Phoenix, etc.): 
Type (provisional, life, etc Q : 



Subject area (Math, Music, etc.): 



Level (primary, elementary, etc,):. 
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********************************************************************** 
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Teacher tackf:round information, page two. 



Teaching experience: 

How many years have you taught English or Language Arts 

at the secondary level in American Samoa? 

How many years have you taught in American Samoa? 

How many years have you taught non-native speakers of 

English in English medium schools? 

How many years have you taught non-native speakers of 

English? 

How many years have you taught at the secondary level? — 

How many years have you taught? 

Please rate your ability to use English and Samoan by the 
descriptors given below. Place an "X" in the highest 
applicable box for each language, 

English Samoan 

Near-native or native ability 

Sufficient to be used as the sole medium 

of instruction ■ 



Sufficient to be used routinely for 

sinple explanations 

Insufficient to be routinely useful in a 
classroom situation 



What other training, qualifications or experience (not covered 
above) do you have that you feel will help you in the secondary 
English program? Please include training, publications, 
supervision, etc. 
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APPENDIX B 



Management and Evaluation Matrix for 
Upgrading Language Ability in American Samoa 
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Summary of Teacher Evaluation 
Of ULA Materials 



Teachers v/ere asked to assess ULA materials by responding to 
the following questions. 



1, Please rate the unit by circling the appropriate number 

A, Too easy - too difficult 

B, Too little vocabulary - too much vocabulary 

C, Too few exercises too many exercises 

D, Unit too long - unit too short 

E, Stories too childish - stories too mature 



2, Please rate the unit by circling the appropriate number 

A, Stories boring - stories interesting 

3, Exercises boring - exercises interesting 

C, Culturally inappropriate - culturally appropriate 

D, Grammar inappropriate - gramnar appropriate 

E, Vocabulary inappropriate - vocabulary appropriate 
?. Instruction not clear - instruction clear 

3, What do you like best about the materials? (Comments) 
4c What do you like least about the materials? (Comments) 

5, Do you have any recommendations for additions, subtrac- 
tions, or changes in the material in either content, 
format, or any other aspects? Please give details. 

6, If you have any other comments not covered in this 
questionnaire that you would like to make about the 
materials, please do so here. 



Summary of the responses to each question are reported on the 
following pagec. 
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HKSF ^;-JK: FREQUENCY: 

3. What do you like best about th^^ materials? 

— culturally appropriate 7 

— very interesting (hig;h interest) 15 

— variety of exercises 6 

— simple enough for students 2 

— good reading level 1 
—kids liked it 

— taught the students something they needed 

to know 2 

— content = stimulating discussion 1 

— content 1 

— the fact that someone els^ prepared them 1 

— stories 2 

— vocabulary exercises 3 

— it was something 1 

— exercises good (esp. multiple choice) 5 

— grammar is easy to understand l\ 
--materials provide lots of teaching 

activities 3 
— students ask lots of questions about 

cultural differences 3 

— taped exercises by teacher l\ 
— exerciser. v;here students have multiple 

choice of verbs l\ 

— number of exercises l\ 

— fill in blank exericses 2 

— more interesting story than past exercises 1 

— being presented with a story 2 



What do you like least about the materials? 

— no pictures 3 

— the vocabulary (exercises) 3 

— repetition of exercises at beginning 2 

— too many sentence combining at beginning 2 

— instructions (T.G-) 2 

— also, five more examples in the instructions 1 

— doesn^t provide enough exercises 2 

— stories boring 1 

— rules and examples don^t get across 1 

— format dull 1 

—not "slick^^ 1 

— need more exercises 6 

— more vocabulary/spelling 1 

— more examples needed 1 

— some excroise.s a bit long 1 

— too long '3r;d difficult (students get bored) 7 

— format - not much open space 1 

--no tape recorder or scripts 2 
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RESFCNSE:: 



FREQUtlNCY: 



exercises difficult to present 1 

more exercises on the easy level 1 

had trouble with #10 1 

need more paraphrase exercises at easier 

level 1 

■too many new words 1 

■length 1 

■no tape recorder 2 



Do you have any recommendations for additions, subtrac- 
tions, or changes in the material in either content, 
format, or any other aspects? Please give details. 



pi ctures 




17 


tape recorder 




5 


more vocabulary and exercises 


(with 




examples ) 




5 


more interesting storiu^ 




2 


give instructions to teachers 


about hov; 




to do exercises 




1 


for this class the exercises 


with the 





factual, historical students were good 

(If Gr, Britain. . . , etc. ) 2 

•more examples that can be acted out in class 1 

■more of exercise #8 type 2 

■more puszles (structure oriented) 4 

■more ? exercise #9 and #10 1 

•exercise #6, too much jargon 1 

•more reading materials 3 

■need tests 1 
■more stories like, "Let's Fall in Love", 

mysteries 5 

-need more 1 

■must have locally developed high interest 

material 1 

■more tape work (exercises) 1 

■more short stories 1 

■pages are too crowded (format) 6 

•tapes made by competent actors 1 

•video tapes 1 

-most of vocabulary is at recognition level 5 

•mere "speaking" exercises 4 

•gamer,, c^^ossword puzzles 3 
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RESPONSE: 



FHEQUENCY: 



If you have any other comments not covered in this 
questionnaire that you would like to make about the 
materials, please do so here. 



tape recorders would be useful, if FHS 

is refurbished 2 

need low level materials 1 

■commercial materials too dull 1 

■need locally developed materials 2 
■teachers should be given more information 

about the purpose for some exercises 

(ex., "Kerens a passive exercise.") 1 

tape recorder 1 
•on usage rule pages, leave outside halC 

blank for students^ notes 1 
■more materials (topics) be about cultural 

differences between U.S. and Samoa 1 
•topics be related to other themes and 

resources available (movies) ^ 

•have story straight (all verbs included) ^ 

•more examples ^ 
►add short anecdotes using same vocabulary 

as stories to reinforce vocabulary 1 

■low leve] ESL class in A.M. 1 

•dictionaries - not enough (Leone) 1 

■use real characters in stories 2 
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APPENDIX D 



Project ULA Class Plan Form 



109 



PROJECT ULA CLASS PLAN 

BRIEF INSTRUC'l WS ARE ON PAGE FOUR. 

Please circle o fill in the appropriate rcisponv.cc> to the follcwhvj 
identification al as si f i cations : 



Offic. uiu 

IX 



SCHOOL: Leone 

Samoana 
Faga ' itua 
Manu ' a 



Yf:AR: Freshman CLASS: L.A. LKVF.L: 1 (F.SL exempt) 

Sophomore ESL 2 (high FSL) 

Junior 3 (mid ESL) 

Senior 4 (low FSL) 



PERIOD: 



BUILDING: 



ROOM: 



TEACHER: 



o 



3 



3 

3 



C 



3 
c 

cr 



n 
r- 
> 

CO 
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DATi: PkBPARBD: 



Complete thic calendar 
ae inatructed during 
Teacher Orientation or, 
if you were unable to 
attend, aek your Dept* 
Head for inetructione * 
Sample calendars are 
available in your 
Teacher *e Resource 
Center. 



MATERIALS AND ACIIVITIES CALEIiDAR 
Week 12 3 4 5 



in 



T 



-\-\-\-\ 



-I- 



-\-\-\-\- 



\ 



\-\ 
}- + 



ALL CLASSES 

Which of the above materials or activities do you expect to be the 
major source of NEW learning for these students? 



What other materials, not on the calendar, do you hope to use during 
the semester? (Reference works, occasional readers, etc.) 



Ill 
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8 9 10 11 12 13 14 15 lb 17 18 

\ 1 \ 1 \ I i \ \ I 



1 - 


1 - 


\ 


- } - 


\ - 


+ 


-i- 


I 


- 1 


-1 


- 1 - 

1 


1 - 


1 - 


1 

1 


- } - 


1 - 


} 


- } - 


1 


- 1 


-1 


- t - 


1 - 


1 - 


1 

T 


- I - 


f- 


I 


-1- 


} 


- \ 


-} 


-\- 


1 - 


\ - 


\ 


- \ - 


1 - 


\ 




1 

t 


- 1 


- } 





ESL CLASSES ONLY 

Indicate briefly how you selected each of the materials or activities 
on the above calendar. Answers such as "EST scores," "advice of last 
year's teacher of these students** or "tryout during first two weeks** 
are sufficient. 



Material 


Basis for selection 
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Please read thena instructions before completing your rlaar. plan.'., 
2 Complete a pl^in in duplicate for each Language Arts or KSL claiu^ that 

2 \Tep^TnJ'copy and submit the other to your Department Head. 

3 The plan rill be reviewed by your Department Head, your Principal, 
and the project staff before being approved on your Prtnctpal s 

4 7nl^7hanles that you wish to make after submitting the plan should be 
made on your Department Head' copy and the project offtce not%}vea 
that a change has been made. 

5 Be sure to keep your Departmer c Head's copy of the class Usts up to 
date. Test score summaries, pre-labelled test answer sheets, and such 
like will be based on this list. 

6 Suggestions for making Class Plans and this form more useful for you 
and your school will be appreciated. 



o 



CO 



0^ 



K) 



o 



K) 



K) 

00 
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to 
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APPENDIX E 



Classroom Observation Checklist 
For ULA Project 
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CLASSROOM OBSERVATION CHECKLIST 
FOR mA PROJECT 



Directions ; 

1, Contact each of the teachers involved a^ad request permission to observe 
their class for at least 30 minutes during the week. Leave uncertain the 
exact date within the week decided upon. 

2, Assure each teacher that your observation is not for the purpose of an 
employment evaluation but is simply to gain an impression of how things 
are going in the ULA program. 

3, Ask each teacher to designate what portion of the day is specifically devoted 
to planned "language arts" or ESL instructional activities. 

4, Observe an hour of inclass activity devoted to ^^ULA" teaching. 

5, Make an unobtnisive record of Ciassroom activities, 

6, Be sure to sample one hour of language arts or ESL instruction for each 
teacher participating in the programs, 

?• Discuss written observations and options for improvement with project staff, 

8, Forward all completed and edited forms to evaluator to be used as one 
record of progress made in upgrading instruction. 
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Section #1 



Name of School 
Date Observed 



month day year 
Name of Observer 



Section #2 

1. Were specially prepared ULA materials utilized during the 
period observed? 

i I a. yes [^2 ^' ^° 

la. If yes, specify type of materials being used. 



1. 
2. 
3. 
4. 
5. 



2. Note the proportion of the hour that the teacher utilized 
ULA material in instruction. 

I I a, 6 minutes or lees 
D b. 7-12 

QJ c. 14 - 18 

d. 19 - 24 

O'- 25 - 30 



ERIC 

hminniirnrfTiaaa 



Did the teacher utilize methods cover' in previous inservice 
sessions . 

I I a , ves Q b. no 

If yes specily type of methods being used, 

1. 

2- 

3, 

4, 



Gauge the amount of intellectual curiosity evidenced by 
pupils in the classroom. Would you characterize the 
atmosphere as being; 

I I a. Highly charged with intellectual curiosity • 

j \ b. Moderately charged with intellectual curiosity, 

I I c* Somewhat charged with intellectual 

j I d. Not very charged with intellectual curiosity. 

Additional observation and comments. 
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APPENDIX F 



English Structures Test 
Test Administration Procedure 
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TEACHER 



ENGLISH STRUCTURES TEST 

TEST ADMINISTRATION PROCEDURE 



CLASS 



PERIOD 

HO. OF TESTS 



1. Have on hand the dozen pencils, sharpened, that you were 
given along with all of your tests. Also, if students 
are not to leave when they finish (see instruction #23) 
have on hand someth:.ng to occupy students who have 
finished the test, such as a worksheet, a book or maga-~ 
zine, etc. 

2. Take roll. 

3. List missing students in the space provided below; iiast 
name first, comma, first name. Please print. 

4. Write on the chalkboard, in large block letters; 
NAME 

DATE May 10, 197^ 

SCHOOL Manuka 

"^''^TEACHER Helene Miller 



OR FAGA^ITUA or SAMOANA or LEuNE as appropriate but do 
not write "High School" since the space on the answer 
sheet is limited. 

''^'^*^ar whatever your name is. Note that we want the name of 
the students^ ESL teacher, not LA teacher or some sub- ^ 
stitute test administrator. Note also that field testing 
indicates that not all students know their teacher's 
name, many know only first or last name, and many can't 
spell either name. 

5, Tell the students that they are going to take an English 
test today. Tell them that it will measure how much 
English they know but that they will NOT be graded on 
the test; that the test is for planning. 

6. Announce the policy on leaving (or not leaving) after 
completing the test (see instruction #23). 



119 



115 



7, Tell the students that you are now going to pass out 
the ansv/er sheets and that they should PRINT their 
COMPLETE name, first name first, second name second, 
and the information that you have put on the blackboard, 
in the appropriate spaces on the answer sheet, (Not ^11 
students know the difference between 'print' and 'write' 
so you nay need to explain*) 

8, Distribute the answer sheets. 

9, Remind students that they are to PRINT, 

10, WALK AROUND THE ROOM AND CHECK THAT STUDENTS ARE FILLING 
IN THE INFORMATION CORRECTLY, 

11, Tell the students that the test books will now be 
distributed and they are NOT to OPEN them until you tell 
them to, 

12, If you trust the students behind your back, pass out the 
tests. If you don't, have a helper or student pass 
them out. Check here if someone else passed out *che 
test; 



13, Tell the students to read the instructions on the front 
cover of the test, work the example, and mark the 
answer of the example in the box labeled. "EXAMPLE" at 
the top of the answer sheet, 

14, Watch that no students open the books or begin working 
the test, 

15, When the students seem to have finished, ask "V>niat 
should Roma say?" Accept "I am fine, thank you," 

16, Say "The answer on your answer sheet should look like 
the answer in the instruction," or words to that effect 
in Samoan or English, 



ENGLISH STRUCTURES TEST, TEST ADMINISTRATION PROCEDURES, 
PAGE TWO 



17, Ask if there are any questions. 

18, When all questions about the mechanics of taking the 
test have been answered, say "WHEN YOU FINISH, RAISE 
YOUR HAND AND I WILL TAKE YOUR TEST AND ANSWER SHEET, 

DO NOT BRING THEM TO ME, NOW OPEN YOUR BOOKS AND BEGIN." 
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19. Students shall have up to 40 minutes to complete the 
test. Write the actual beginning time in the first 
box below. Add 40 minutes and write the last possible 
finishing time in the second box below. 




3egin 



Finish 



If any students have not finished the test by the 
finishing time that you have calculated above, take 
their papers from them aud write "unfinished" above the 
"example" box. Do not worry if the last possible 
finishing time is after the class period is supposed 
to end. In field testing the slowest students took 
32 minutes. Allow any student the full 40 minutes if 
he wants it even if that goes over into the next period. 

20. If any student arrives during the instructions, give 
them the papers that the rest of the class has at that 
time and help him to catch up. If any student arrives 
after the actual timed test has begun^ do not 
administer the test to him; treat him aG a student who 
has finished (see inst. #23 below / . 

21. During the test, walk around the room watching the 
students. If any students are obviously cheating, put 
an ^X^ in the box in the lower right hand corner of 
their answer sheets but do not take their tests away. 

22. When students finish and raise their hands, take their 
tests and answer sheets, check that all information 
has been included and that the answer sheet has been 
used correctly, underline the student ^s surname (second.* 
name) on his answer sheet. 

23. If school policy permits, students should be excused 
from class when they finish. If not, they should be 
required to stay quietly at their desks. If no policy 
has been established, consideration should be given 

to the fact that at Samoana and Faga'itua some students 
will have to pass through other classrooms if th<=y are 
excused. 

24. If some students bring their test book and answer sheet 

to you, remind the whole class that they are to raise their 
hand and wait for you to collect their papers. 
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25. When all students have finished, count the test books 
and answer sheets. You should have used and unused tests 
tests and answer sheets totaling the figure in the box 
at the beginning of these instructions. Check that 

you have underlined all surnames. 

26. Repackage the materials in the envelopes that they came 
in with these instructions around the outside and return 
them to the Project test supervisor at your school. 



MISSING STUDENTS: 



Please note on the back of this sheet, any disturbances 
that occurred during the test (such aa pii«amatic hammers 
outside the window, students passing through the room, etc ) . 
Also please make any suggestions or criticisms that you feel 
would help make future test administrations better. 
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APPENDIX G 
Test Data 
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TABLE 21 



Distribution Statistics and Standard Scores 



Number of Valid 
Scores: 13^^^ 
Mean: 27.73 
Standard Deviation: 
9.11 

Skewness : 

-1.23 P 0.2253 
Kurtosis : 

-h.^i2 P 0.0001 



ERIC 



1373 EST 
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Norms 
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Percentile 
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'7.0000 


2 


0 


99 


71 
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TABLE 22 



Distribution Statistics and Standard Scores 
197^ EST Local Norms 



:iumter of Valid 
Scores: 1379 
Mean: 30.16 
Standard Deviation: 
10.03 

Skewness : 

-0.7225 P 0.5229 
Kurtosis : 

-6.7717 P 0.0000 
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11.0000 


9 


1 


2 


31 
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